This paper introduces a deep reinforcement learning approach to solve the virtual network function scheduling problem in dynamic scenarios. We formulate an integer linear programming model for the problem in static scenarios. In dynamic scenarios, we define the state, action, and reward to form the learning approach. The learning agents are applied with the asynchronous advantage actor-critic algorithm. We assign a master agent and several worker agents to each network function virtualization node in the problem. The worker agents work in parallel to help the master agent make decision. We compare the introduced approach with existing approaches by applying them in simulated environments. The existing approaches include three greedy approaches, a simulated annealing approach, and an integer linear programming approach. The numerical results show that the introduced deep reinforcement learning approach improves the performance by 6-27% in our examined cases.

This paper investigates a service deployment model for network function virtualization which handles per-flow priority to minimize the deployment cost. Service providers need to implement network services each of which consists of one or more virtual network functions (VNFs) with satisfying requirements of service delays. In our previous work, we studied the service deployment model with per-host priority; flows belonging to the same service, for the same VNF, and handled on the same host have the same priority. We formulated the model as an optimization problem, and developed a heuristic algorithm named FlexSize to solve it in practical time. In this paper, we address per-flow priority, in which flows of the same service, VNF, and host have different priorities. In addition, we expand FlexSize to handle per-flow priority. We evaluate per-flow and per-host priorities, and the numerical results show that per-flow priority reduces deployment cost compared with per-host priority.

This paper proposes a backup resource allocation model for virtual network functions (VNFs) to minimize the total allocated computing capacity for backup with considering the service delay. If failures occur to primary hosts, the VNFs in failed hosts are recovered by backup hosts whose allocation is pre-determined. We introduce probabilistic protection, where the probability that the protection by a backup host fails is limited within a given value; it allows backup resource sharing to reduce the total allocated computing capacity. The previous work does not consider the service delay constraint in the backup resource allocation problem. The proposed model considers that the probability that the service delay, which consists of networking delay between hosts and processing delay in each VNF, exceeds its threshold is constrained within a given value. We introduce a basic algorithm to solve our formulated delay-constraint optimization problem. In a problem with the size that cannot be solved within an acceptable computation time limit by the basic algorithm, we develop a simulated annealing algorithm incorporating Yen's algorithm to handle the delay constraint heuristically. We observe that both algorithms in the proposed model reduce the total allocated computing capacity by up to 56.3% compared to a baseline; the simulated annealing algorithm can get feasible solutions in problems where the basic algorithm cannot.

Mission-critical monitoring services, such as finding criminals with a monitoring camera, require rapid detection of newly updated data, where suppressing delay is desirable. Taking this direction, this paper proposes a network design scheme to minimize this delay for monitoring services that consist of Internet-of-Things (IoT) devices located at terminal endpoints (TEs), databases (DB), and applications (APLs). The proposed scheme determines the allocation of DB and APLs and the selection of the server to which TE belongs. DB and APL are allocated on an optimal server from multiple servers in the network. We formulate the proposed network design scheme as an integer linear programming problem. The delay reduction effect of the proposed scheme is evaluated under two network topologies and a monitoring camera system network. In the two network topologies, the delays of the proposed scheme are 78 and 80 percent, compared to that of the conventional scheme. In the monitoring camera system network, the delay of the proposed scheme is 77 percent compared to that of the conventional scheme. These results indicate that the proposed scheme reduces the delay compared to the conventional scheme where APLs are located near TEs. The computation time of the proposed scheme is acceptable for the design phase before the service is launched. The proposed scheme can contribute to a network design that detects newly added objects quickly in the monitoring services.

This paper proposes a virtual network function (VNF) placement model considering both availability and probabilistic protection for the service delay to minimize the service deployment cost. Both availability and service delay are key requirements of services; a service provider handles the VNF placement problem with the goal of minimizing the service deployment cost while meeting these and other requirements. The previous works do not consider the delay of each route which the service can take when considering both availability and delay in the VNF placement problem; only the maximum delay was considered. We introduce probabilistic protection for service delay to minimize the service deployment cost with availability. The proposed model considers that the probability that the service delay, which consists of networking delay between hosts and processing delay in each VNF, exceeds its threshold is constrained within a given value; it also considers that the availability is constrained within a given value. We develop a two-stage heuristic algorithm to solve the VNF placement problem; it decides primary VNF placement by solving mixed-integer second-order cone programming in the first stage and backup VNF placement in the second stage. We observe that the proposed model reduces the service deployment cost compared to a baseline that considers the maximum delay by up to 12%, and that it obtains a feasible solution while the baseline does not in some examined situations.

Multi-core fiber (MCF) is one of the promising space-division multiplexing technologies to increase the capacity of optical networks. MCF-based networks have two challenges. One is the inter-core crosstalk (XT) that degrades the quality of optical signals in two neighboring fiber cores. The other is network protection against link failures that cause massive data loss. One way to protect against multiple link failures is to prepare physically separated links as a backup network. Probabilistic protection improves the efficiency of protection by allowing a certain probability of protection failure. Existing studies on backup network design with probabilistic protection do not target MCF-based networks, which raises problems such as protection failure due to the inter-core XT and excessive consumption of optical resources. To address these problems, this paper proposes a XT-aware backup network design model for the MCF optical path networks. The proposed model protects the network against probabilistic multiple link failures. We adopt probabilistic protection that allows a certain probability of protection failure due to the inter-core XT and minimizes the required number of links in the backup network. We present an algorithm to satisfy the probabilistic protection requirement and formulate the model as an integer linear programming problem. We develop a heuristic approach to apply the proposed model to larger networks. Numerical results observe that the proposed model requires fewer links than the dedicated allocation model, which provisions the backup paths in the same manner as the primary paths.

When mission-critical applications are provided over a network, high availability is required in addition to a low delay. This paper proposes a multi-homing network design model, named MHND, that achieves low delay, high availability, and the order guarantee of events. MHND maintains the event occurrence order with a multi-homing configuration using conservative synchronization. We formulate MHND as an integer linear programming problem to minimize the delay. We prove that the distributed server allocation problem with MHND is NP-complete. Numerical results indicate that, as a multi-homing number, which is the number of servers to which each user belongs, increases, the availability increases while increasing the delay. Noteworthy, two or more multi-homing can achieve approximately an order of magnitude higher availability compared to that of conventional single-homing at the expense of a delay increase up to two times. By using MHND, flexible network design is achieved based on the acceptable delay in service and the required availability.

This paper introduces heuristic approaches and a deep reinforcement learning approach to solve a joint virtual network function deployment and scheduling problem in a dynamic scenario. We formulate the problem as an optimization problem. Based on the mathematical description of the optimization problem, we introduce three heuristic approaches and a deep reinforcement learning approach to solve the problem. We define an objective to maximize the ratio of delay-satisfied requests while minimizing the average resource cost for a dynamic scenario. Our introduced two greedy approaches are named finish time greedy and computational resource greedy, respectively. In the finish time greedy approach, we make each request be finished as soon as possible despite its resource cost; in the computational resource greedy approach, we make each request occupy as few resources as possible despite its finish time. Our introduced simulated annealing approach generates feasible solutions randomly and converges to an approximate solution. In our learning-based approach, neural networks are trained to make decisions. We use a simulated environment to evaluate the performances of our introduced approaches. Numerical results show that the introduced deep reinforcement learning approach has the best performance in terms of benefit in our examined cases.

Mobile edge computing (MEC) is a key technology for providing services that require low latency by migrating cloud functions to the network edge. The potential low quality of the wireless channel should be noted when mobile users with limited computing resources offload tasks to an MEC server. To improve the transmission reliability, it is necessary to perform resource allocation in an MEC server, taking into account the current channel quality and the resource contention. There are several works that take a deep reinforcement learning (DRL) approach to address such resource allocation. However, these approaches consider a fixed number of users offloading their tasks, and do not assume a situation where the number of users varies due to user mobility. This paper proposes Deep reinforcement learning model for MEC Resource Allocation with Dummy (DMRA-D), an online learning model that addresses the resource allocation in an MEC server under the situation where the number of users varies. By adopting dummy state/action, DMRA-D keeps the state/action representation. Therefore, DMRA-D can continue to learn one model regardless of variation in the number of users during the operation. Numerical results show that DMRA-D improves the success rate of task submission while continuing learning under the situation where the number of users varies.

This paper proposes a network design model, considering data consistency for a delay-sensitive distributed processing system. The data consistency is determined by collating the own state and the states of slave servers. If the state is mismatched with other servers, the rollback process is initiated to modify the state to guarantee data consistency. In the proposed model, the selected servers and the master-slave server pairs are determined to minimize the end-to-end delay and the delay for data consistency. We formulate the proposed model as an integer linear programming problem. We evaluate the delay performance and computation time. We evaluate the proposed model in two network models with two, three, and four slave servers. The proposed model reduces the delay for data consistency by up to 31 percent compared to that of a typical model that collates the status of all servers at one master server. The computation time is a few seconds, which is an acceptable time for network design before service launch. These results indicate that the proposed model is effective for delay-sensitive applications.

Link duplication is widely used in Internet protocol networks to tackle the network congestion increase caused by link failure. Network congestion represents the highest link utilization over all the links in the network. Due to capital expenditure constraints, not every link can be duplicated to reduce congestion after a link fails. Giving priority to some selected links makes sense. Meanwhile, traffic routes are determined by link weights that are configured in advance. Therefore, choosing an appropriate set of link weights reduces the number of links that actually need to be duplicated in order to keep a manageable congestion under failure. A manageable congestion is a congestion under which Service Level Agreements can be met. The conventional scheme fixes link weights before determining links to duplicate. In this scheme, the fixed link weights are optimized to minimize the worst network congestion. The worst network congestion is the highest network congestion over all the single non-duplicated link failures. As the selection of links for protection depends on the fixed link weights, some suitable protection patterns, which are not considered with other possible link weights, might be skipped leading to overprotection. The paper proposes a scheme that considers multiple protection scenarios before optimizing link weights in order to reduce the overall number of protected links. Simulation results show that the proposed scheme uses fewer link protections compared to the conventional scheme.

Predictive spatial-monitoring, which predicts spatial information such as road traffic, has attracted much attention in the context of smart cities. Machine learning enables predictive spatial-monitoring by using a large amount of aggregated sensor data. Since the capacity of mobile networks is strictly limited, serious transmission delays occur when loads of communication traffic are heavy. If some of the data used for predictive spatial-monitoring do not arrive on time, prediction accuracy degrades because the prediction has to be done using only the received data, which implies that data for prediction are ‘delay-sensitive’. A utility-based allocation technique has suggested modeling of temporal characteristics of such delay-sensitive data for prioritized transmission. However, no study has addressed temporal model for prioritized transmission in predictive spatial-monitoring. Therefore, this paper proposes a scheme that enables the creation of a temporal model for predictive spatial-monitoring. The scheme is roughly composed of two steps: the first involves creating training data from original time-series data and a machine learning model that can use the data, while the second step involves modeling a temporal model using feature selection in the learning model. Feature selection enables the estimation of the importance of data in terms of how much the data contribute to prediction accuracy from the machine learning model. This paper considers road-traffic prediction as a scenario and shows that the temporal models created with the proposed scheme can handle real spatial datasets. A numerical study demonstrated how our temporal model works effectively in prioritized transmission for predictive spatial-monitoring in terms of prediction accuracy.

This letter proposes a scheme to update metrics without loops while minimizing routing instability time in an Open Shortest Path First (OSPF) network. The original OSPF network enters the transient state when metrics are being updated to improve the routing performance, and in this state packets may fall into loops. This may cause packet loss and inefficient network resource utilization. To avoid transient loops, a conventional scheme gives each router a priority that reflects the optimum time for metric update. However, when the updated metrics include both larger and smaller values than the preceding ones, two sequential updating processes, one for larger values and one for smaller values, are required. It takes time to converge on the final metric values in the conventional scheme, given that the interval time between the two processes is not insignificant. The second process starts only when the first process is confirmed to be completed. The interval time including the confirmation time and the time needed to reconfigure the metrics in all routers, lengthens the transient state duration; from several seconds to several tens of seconds. This causes routing instability. The proposed scheme transforms the set of updated metrics into an equivalent set of metrics that are either all larger or all smaller (if changed at all) than the ones before the update. The set of equivalent metrics yield exactly the same results in terms of routing as the conventional scheme, i.e. the result desired by the network operator. The non-mixture update requires only one updating process and so eliminates the interval time. Numerical results indicate that the probability that the proposed scheme can achieve non-mixture update is more than 67% in the networks examined.

A flexible orthogonal frequency-division multiplexing optical network enables the bandwidth to be flexibly changed by changing the number of sub-carriers. We assume that users request to dynamically change the number of sub-carriers. Dynamic bandwidth changes allow the network resources to be used more efficiently but each change takes a significant amount of time to complete. Service centric resource allocation must be considered in terms of the waiting time needed to change the number of sub-carriers. If the user demands drastically increase such as just after a disaster, the waiting time due to a chain-change of bandwidth becomes excessive because disaster priority telephone services are time-critical. This paper proposes a Grouped-elastic spectrum allocation scheme to satisfy the tolerable waiting time of the service in an optical fiber link. Spectra are grouped to restrict a waiting time in the proposed scheme. In addition, the proposed scheme determines a bandwidth margin between neighbor spectra to spectra to prevent frequent reallocation by estimating real traffic behavior in each group. Numerical results show that the bandwidth requirements can be minimized while satisfying the waiting time constraints. Additionally measurement granularity and channel alignment are discussed.

Cost-effective cloud storage services are attracting users with their convenience, but there is a trade-off between service availability and usage cost. We develop two cloud provider selection models for cloud storage services to minimize the total cost of usage. The models select multiple cloud providers to meet the user requirements while considering unavailability. The first model, called a user-copy (UC) model, allows the selection of multiple cloud providers, where the user copies its data to multiple providers. In addition to the user copy function of the UC model, the second model, which is called a user and cloud-provider copy (UCC) model, allows cloud providers to make copies of the data to deliver them to other cloud providers. The cloud service is available if at least one cloud provider is available. We formulate both models as integer linear programming (ILP) problems. Our performance evaluation observes that both models reduce the total cost of usage, compared to the single cloud provider selection approach. As the cost of bandwidth usage between a user and a cloud provider increases, the UCC model becomes more beneficial than the UC model. We implement the prototype for cloud storage services, and demonstrate our models via Science Information Network 5.

The machine-to-machine (M2M) service network platform that accommodates and controls various types of Internet of Things devices has been presented. This paper investigates program file placement strategies for the M2M service network platform that achieve low blocking ratios of new task requests and accommodate as many tasks as possible in the dynamic scenario. We present four strategies for determining program file placement, which differ in the computation method and whether the relocation of program files being used by existing tasks is allowed or not. Simulation results show that a strategy based on solving a mixed-integer linear programming model achieves the lowest blocking ratio, but a heuristic algorithm-based strategy can be an attractive option by allowing recomputation of the placement when the placement cannot be obtained at the timing of new task request arrival.

New network survivability measures are developed and compared with conventional ones. The advantages of using multiple survivability measures, including the new ones, are discussed. The measures are illustrated and interpreted through several numerical examples. We also show how survivability can be included as a constraint in network optimization models.

This paper proposes a high-speed connection admission control scheme, named PERB CAC (CAC based on Prior Estimation for Residual Bandwidth). This scheme estimates the residual bandwidth in advance by generating a series of virtual requests for connection. When an actual connection request occurs, PERB CAC can instantaneously judge if the required bandwidth is larger than the estimated residual bandwidth, so the connection set-up time can be greatly reduced. Therefore, PERB CAC can realize high-speed connection set-up.

This paper proposes two algorithms, namely Server-User Matching (SUM) algorithm and Extended Server-User Matching (ESUM) algorithm, for the distributed server allocation problem. The server allocation problem is to determine the matching between servers and users to minimize the maximum delay, which is the maximum time to complete user synchronization. We analyze the computational time complexity. We prove that the SUM algorithm obtains the optimal solutions in polynomial time for the special case that all server-server delay values are the same and constant. We provide the upper and lower bounds when the SUM algorithm is applied to the general server allocation problem. We show that the ESUM algorithm is a fixed-parameter tractable algorithm that can attain the optimal solution for the server allocation problem parameterized by the number of servers. Numerical results show that the computation time of ESUM follows the analyzed complexity while the ESUM algorithm outperforms the approach of integer linear programming solved by our examined solver.

This letter proposes a practical scheme that can estimate ADSL link rates. The proposed scheme allows us to estimate ADSL link rates from measurements made at the NOC using existing communications protocols and network node facilities; it imposes no heavy traffic overhead. The proposed scheme consists of two major steps. The first step is to collect measured data of round trip times (RTT) for both long and short packets to find their minimum values of RTTs by sending Internet Control Message Protocol (ICMP) echo request messages. The second step is to estimate the ADSL down- and up-link rates by using the difference in RTT between long and short packets and the experimentally-obtained correlated relationships between ADSL down- and up-link rates. RTTs are experimentally measured for an IP network, and it is shown that the down- and up-link rates can be obtained in a simple manner.